Techniques for the operation and control of robotic manipulators in some environments, such as manufacturing facilities, have been developed and used successfully for many years. Many such techniques include the creation of new computer programs for each new robotic task.
These techniques must be substantially modified for use in poorly modeled environments, or to perform tasks in response to unplanned scenarios and when there is a potentially substantial time delay between command generation and task execution as occurs for example in surface controlled undersea and ground controlled space robotic operations.
One convention approach would be to develop an interpretive robot language which can then be used to write a program which will execute on the robot to execute a specific task. The program would be sent to the system executive which would execute the program to control the robot. The interpretive language approach may be desirable for use in factory automation and similar tasks where a specialized algorithm may be needed for each task.
One major improvement in this area has been the development of systems, as described for example in U.S. patent application Ser. No. 07/699,299, filed May 9, 1991 by the inventor hereof, in which series of task primitive parameters are transmitted between control and operation locations to provide task execution control by one or more task execution primitives and obviate the need to prepare and transmit new robotic control programs for each new task.
The task execution primitive approach is particularly suited for space telerobotics applications where flight qualifying software is required. In this case, the software for the task execution system which resides in space is fixed and can be completely flight qualified before the missions. Specific applications are then specified by the new set of task execution parameters which are transmitted to the space vehicle to describe the desired robot behavior without the need for new programming.
A task execution primitive may be described as a function which controls a manipulator to perform the task described by its input parameter set. The primitive generates the desired setpoints and performs the desired control. The parameter list is the interface between a higher level task planning system and task execution. The details of the implementation are hidden from the planning system. The planning system only needs to know how to describe the desired behavior of execution by setting the input parameters of the task primitive. The command to execute the task primitive and the input parameter list are received from the planning system by a system execute which starts execution of the primitive and returns periodic execution status reports.
In general, remote control of such robotic operation may be accomplished by teleoperation, autonomous or supervisory control as well as a combination of these approaches which is known as shared control.
Interactive robotic task planning, execution and monitoring can be accomplished with pure teleoperation. In this approach, planning resides within the operator's mind, execution is issued by the operator via hand controllers and monitoring is provided by sensory feedback to the operator. Autonomous task planning, execution, and monitoring is the other extreme to teleoperation. Here, the operator initiates only very high level commands such as "replace the electronics module" and planning, execution, and monitoring is then done autonomously without further operator input.
Teleoperation has proven to be a valuable tool for many tasks especially in unmodeled or poorly modeled environments and for unplanned scenarios. The increasing complexity of the tasks to be performed places an ever increasing burden on the operator. Autonomous control is becoming increasingly valuable as a tool to relieve the operator of many task planning, execution, and monitoring responsibilities in order to allow the operator to concentrate on the more crucial elements of a task.
Supervisory and shared control are recent improvements in telerobot task execution for unplanned scenarios, or for poorly modeled environments. Supervisory control is where the operator selects autonomous control commands and associated parameterization for a task and can stop execution at any time. Shared control is the mixing of inputs from an operator and an autonomous control system during task execution.
A key element needed for planning, execution, and monitoring in a supervisory or shared control system is an operator interface which supports shared and supervisory control features. Supervisory features are required to permit the operator to set up teleoperation, autonomous, and shared control task environment parameters and to provide specific input parameters for autonomous task primitives and teleoperation control.
Supervisory and shared control systems benefit greatly from the use of task primitives, which are reusable, predetermined, self contained, preprogrammed programs for controlling a robot to accomplish various tasks, such control being dependent upon a set of input parameters which may be designated before, at the beginning time or during task execution. Shared features of an operator interface are required in order to provide autonomous setting of some environment and control parameters depending upon task context.
The utility of a particular task primitive depends upon it's flexibility. The utilization of sensors, both real and virtual, enhances task execution capability both by providing alternate approaches for executing the task and by making task execution more robust.
A very simple robotic system might have purely position control of a robot from a trajectory generator. Adding a hand controller allows the operator to perform position teleoperation. A force-torque sensor makes force/compliance control possible and therefore robust contact tasks. A virtual force field sensor can aid the operator during teleoperation to keep the robot away from joint limits and objects. Individual task primitives may be provided in the remote robot system for each required task, but the limited size of such remote robot systems, in terms of computer memory limitations and/or number of line of programming code that may be incorporated therein, limits the number of such specialized task primitives which may be used.
It is often difficult to include multiple sensors in a robot control system due to the added system complexity, the difficulty in programming the robot to utilize the sensor, and the difficulty in specifying the task to utilize the sensor. What are needed are task execution primitives which simplify task description and execution when utilizing multiple sensory inputs in addition to trajectory information, especially task execution primitives for the planning, execution and monitoring of telerobot tasks in poorly modeled environments and for unplanned scenarios. Such task execution primitives should efficiently and conveniently permit the combination of teleoperation, autonomous, supervisory, and shared control techniques.